<!DOCTYPE html>
<html lang="zh">
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
        <title>cs229 第二节 ( TA ) | Dirac Lee</title><meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="robots" content="noodp" />
<meta name="Description" content="cs229 第二节"><link rel="prev" href="https://diraclee.gitee.io/2020/07/design_pattern/" /><link rel="next" href="https://diraclee.gitee.io/2020/07/cs229_lecture3/" /><link rel="canonical" href="https://diraclee.gitee.io/2020/07/cs229_lecture2/" />
<link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" />
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
<link rel="manifest" href="/site.webmanifest">
<link rel="mask-icon" href="/safari-pinned-tab.svg" color="#5bbad5">
<meta name="msapplication-TileColor" content="#da532c">
<meta name="theme-color" content="#ffffff"><meta property="og:title" content="cs229 第二节 ( TA )" />
<meta property="og:description" content="cs229 第二节" />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://diraclee.gitee.io/2020/07/cs229_lecture2/" />
<meta property="article:published_time" content="2020-07-19T00:00:00+00:00" />
<meta property="article:modified_time" content="2020-07-23T00:00:00+00:00" />
<script type="application/ld+json">
    {
        "@context": "http://schema.org",
        "@type": "BlogPosting",
        "headline": "cs229 第二节 ( TA )",
        "mainEntityOfPage": {
            "@type": "WebPage",
            "@id": "https:\/\/diraclee.gitee.io\/2020\/07\/cs229_lecture2\/"
        },"image": {
                "@type": "ImageObject",
                "url": "https:\/\/diraclee.gitee.io\/logo.png",
                "width":  800 ,
                "height":  600 
            },"genre": "posts","keywords": "机器学习, cs229","wordcount":  1134 ,
        "url": "https:\/\/diraclee.gitee.io\/2020\/07\/cs229_lecture2\/","datePublished": "2020-07-19T00:00:00\x2b00:00","dateModified": "2020-07-23T00:00:00\x2b00:00","license": "This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.","publisher": {
                "@type": "Organization",
                "name": "xxxx",
                "logo": {
                "@type": "ImageObject",
                "url": "https:\/\/diraclee.gitee.io\/logo.png",
                "width":  127 ,
                "height":  40 
                }
            },"description": "cs229 第二节"
    }
    </script><link rel="stylesheet" href="/css/style.min.css"><link rel="stylesheet" href="/css/lib/fontawesome-free/all.min.css"><link rel="stylesheet" href="/css/lib/animate/animate.min.css"></head>
    <body><script>
            window.isDark = (window.localStorage && window.localStorage.getItem('theme')) === 'dark';
            window.isDark && document.body.classList.add('dark-theme');
        </script><div class="wrapper"><nav class="navbar">
    <div class="navbar-container">
        <div class="navbar-header animated bounceIn">
            <a href="https://diraclee.gitee.io">Dirac Lee</a>
        </div>
        <div class="navbar-menu"><a class="menu-item" href="https://diraclee.gitee.io/posts" title="">文章</a><a class="menu-item" href="https://diraclee.gitee.io/tags" title="">标签</a><a class="menu-item" href="https://diraclee.gitee.io/categories" title="">分类</a><a class="menu-item" href="https://diraclee.gitee.io/about" title="">关于</a><a class="menu-item" href="https://diraclee.gitee.io" title="English"><i class="fas fa-language fa-fw"></i></a><a href="javascript:void(0);" class="theme-switch"><i class="fas fa-adjust fa-rotate-180 fa-fw" title="切换主题"></i></a>
        </div>
    </div>
</nav><nav class="navbar-mobile">
    <div class="navbar-container">
        <div class="navbar-header">
            <div class="navbar-header-title animated bounceIn">
                <a href="https://diraclee.gitee.io">Dirac Lee</a>
            </div>
            <div class="menu-toggle" id="menu-toggle">
                <span></span><span></span><span></span>
            </div>
        </div>
        <div class="navbar-menu" id="mobile-menu"><a class="menu-item" href="https://diraclee.gitee.io/posts" title="">文章</a><a class="menu-item" href="https://diraclee.gitee.io/tags" title="">标签</a><a class="menu-item" href="https://diraclee.gitee.io/categories" title="">分类</a><a class="menu-item" href="https://diraclee.gitee.io/about" title="">关于</a><a class="menu-item" href="https://diraclee.gitee.io" title="English"></a><a href="javascript:void(0);" class="theme-switch"><i class="fas fa-adjust fa-rotate-180 fa-fw" title="切换主题"></i></a>
        </div>
    </div>
</nav><main class="main">
                <div class="container"><article class="page"><h1 class="post-title animated flipInX">cs229 第二节 ( TA )</h1><div class="post-meta">
            <div class="post-meta-main"><a class="author" href="https://diraclee.gitee.io" rel="author" target="_blank">
                    <i class="fas fa-user-circle fa-fw"></i>Dirac Lee
                </a>&nbsp;<span class="post-category">收录于&nbsp;<i class="far fa-folder fa-fw"></i><a href="https://diraclee.gitee.io/categories/%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/">学习笔记</a>&nbsp;</span></div>
            <div class="post-meta-other"><i class="far fa-calendar-alt fa-fw"></i><time datetime=2020-07-19>2020-07-19</time>&nbsp;
                <i class="fas fa-pencil-alt fa-fw"></i>约 1134 字&nbsp;
                <i class="far fa-clock fa-fw"></i>预计阅读 3 分钟&nbsp;</div>
        </div><div class="post-content"><p>线性代数</p>
<ul>
<li>术语</li>
<li>基本操作</li>
<li>线性代数对机器学习的意义</li>
<li>几何解释</li>
<li>正交性与投影</li>
<li>线性回归</li>
<li>特征值与特征向量</li>
<li>奇异值分解 (SVD) 与特征分解(Eigen Devision)</li>
<li>谱</li>
<li>正负确定性</li>
</ul>
<a class="post-dummy-target" id="术语"></a><h2>术语</h2>
<p>向量 $v \in \R^n$ 即 $v = \begin{bmatrix} v_1 \\  v_2 \\ \vdots \\  v_n \end{bmatrix}$</p>
<p>矩阵 $A \in \R^{m \times n}$ 即 $A = 
\begin{bmatrix} 
a_{11} &amp; a_{12} &amp; \dots &amp; a_{1n} \\ 
a_{21} &amp; a_{22} &amp; \dots &amp; a_{2n} \\ 
\vdots &amp; \vdots &amp; \dots &amp; \vdots \\ 
a_{m1} &amp; a_{m2} &amp; \dots &amp; a_{mn} 
\end{bmatrix}$</p>
<p>单位矩阵 $I \in \R^{n \times n}$ 则 $I = 
\begin{bmatrix} 
1      &amp;        &amp;        &amp;        \\ 
&amp; 1      &amp;        &amp;        \\ 
&amp;        &amp; \ddots &amp;        \\ 
&amp;        &amp;        &amp; 1 
\end{bmatrix}$</p>
<p>对角矩阵 $D \in \R^{n \times n}$ 则 $D = 
\begin{bmatrix} 
c_1    &amp;        &amp;        &amp;        \\ 
&amp; c_2    &amp;        &amp;        \\ 
&amp;        &amp; \ddots &amp;        \\ 
&amp;        &amp;        &amp; c_n 
\end{bmatrix}$</p>
<p>对称矩阵 $A^T = A$ ，其中 $A^T 
= [a_{ji}]= 
\begin{bmatrix} 
a_{11} &amp; a_{21} &amp; \dots &amp; a_{m1} \\ 
a_{12} &amp; a_{22} &amp; \dots &amp; a_{m2} \\ 
\vdots &amp; \vdots &amp; \dots &amp; \vdots \\ 
a_{1n} &amp; a_{2n} &amp; \dots &amp; a_{mn} 
\end{bmatrix} \in \R^{n \times m}$</p>
<p>矩阵A的迹 $Trace(A) = \sum_i A_{ii}$</p>
<a class="post-dummy-target" id="基本操作"></a><h2>基本操作</h2>
<a class="post-dummy-target" id="向量内积"></a><h3>向量内积</h3>
<p>对于向量 $v \in \R^n, u \in \R^n$</p>
<p>它们的内积 $\begin{aligned} \left&lt;u, v \right&gt; = u^T v = v^T u = \sum_{i=1}^n v_i u_i \in \R \end{aligned}$</p>
<p>如果内积为0，则两个向量正交 $\left&lt;u, v \right&gt; = 0 \implies u \perp v$</p>
<a class="post-dummy-target" id="向量外积"></a><h3>向量外积</h3>
<p>对于向量 $v \in \R^m, u \in \R^n$
它们的外积 $v u^T$ 的秩为 1</p>
<p>对于k对向量 $x^{(i)} \in \R^m, y^{(i)} \in \R^n, i = 1, 2, &hellip; k$</p>
<p>它们外积之和的秩 $r( \sum_{i=1}^k x^{(i)} y^{(i)} ) \le \min \lbrace m, n, k \rbrace$</p>
<a class="post-dummy-target" id="矩阵乘向量略"></a><h3>矩阵乘向量(略)</h3>
<a class="post-dummy-target" id="矩阵乘矩阵略"></a><h3>矩阵乘矩阵(略)</h3>
<p>乘积结果维度分析方法</p>
<ul>
<li>
<p>内积分析：$\left&lt;k行, k列 \right&gt; \in \R$, 结果为每行n个$\R$标量，共n行的拼接</p>
</li>
<li>
<p>外积分析: $m列 \times n 行 \in \R^{m \times n}$, 结果为若干个 $\R^{m \times n}$ 矩阵相加</p>
</li>
</ul>
<p>对于任意矩阵 $A \in \R^{m \times n}$ 有 $A^T A \in \mathbb S^{n \times n}$，其中 $\mathbb S^{n \times n}$ 是指 $n \times n$ 的对称矩阵</p>
<a class="post-dummy-target" id="线性代数对机器学习的意义"></a><h2>线性代数对机器学习的意义</h2>
<a class="post-dummy-target" id="1-表示数据"></a><h3>1. 表示数据</h3>
<p>$X \in \R^{m \times n}$，其中 m 表示样本数量，n 表示特征维度</p>
<a class="post-dummy-target" id="2-概率表示方便"></a><h3>2. 概率表示方便</h3>
<p>向量 $\theta$ 每一个元素都随机初始化，该向量均值可表示为一个向量，方差可表示为一个对称矩阵</p>
<a class="post-dummy-target" id="3-微积分计算方便"></a><h3>3. 微积分计算方便</h3>
<table>
<thead>
<tr>
<th>函数形式</th>
<th>函数值 f</th>
<th>一阶导数 f&rsquo;</th>
<th>二阶导数 f&rsquo;&rsquo;</th>
</tr>
</thead>
<tbody>
<tr>
<td>$f: \R \rightarrow \R $</td>
<td>$\R$</td>
<td>$\R$</td>
<td>$\R$</td>
</tr>
<tr>
<td>$f: \R^n \rightarrow \R $</td>
<td>$\R$</td>
<td>$\R^n$ (Gradient)</td>
<td>$\mathbb {S}^{n \times n}$ (Hessian)</td>
</tr>
<tr>
<td>$f: \R^m \rightarrow \R^n $</td>
<td>$\R^n$</td>
<td>$\R^{m \times n}$ (Jacobian)</td>
<td>$\R^{m \times n \times n}$ (Tensor)</td>
</tr>
</tbody>
</table>
<a class="post-dummy-target" id="4-内核技术"></a><h3>4. 内核技术</h3>
<p>核矩阵 $K \in \mathbb S^{m \times m}$</p>
<a class="post-dummy-target" id="矩阵操作的几何解释"></a><h2>矩阵操作的几何解释</h2>
<p>$A \in \R^{m \times n}$，$x \in \R^n$ 则 $b = Ax \in \R^m$</p>
<p>若记函数 $A(x) = Ax$</p>
<p>则 $A: \R^n \rightarrow \R^m$</p>
<p>例如</p>
<p>$A: \R^3 \rightarrow \R^3$ 表示函数 $A$ 将某三维空间中的点 $x$ 映射到另一三维空间</p>
<p>若 $r(A) = 3$，即 $A$ 满秩，则该映射是双射，即目标空间中的点也可以经过某种方式 ($A^{-1}$) 映射到原空间</p>
<p>若 $r(A) = 2$，则映射所得的点全部分布在该三维空间中的某个二维平面上，不在该二维平面上的点无法映射到源空间</p>
<p>只有矩阵行列数相同且满秩时，该矩阵才是可逆的</p>
<a class="post-dummy-target" id="正交与投影"></a><h2>正交与投影</h2>
<p>$proj(\vec b; \vec v) = 
\begin{bmatrix}
\vec v \vec v^T \\ 
\\ 
\vec v^T \vec v
\end{bmatrix} \vec b = 
\vec{\hat b}$</p>
<p>其中 $\begin{bmatrix}
\vec v \vec v^T \\ 
\\ 
\vec v^T \vec v
\end{bmatrix}$ 称为投影矩阵</p>
<p>令 $V = 
\begin{bmatrix}
|            &amp; |            \\ 
\vec v^{(1)} &amp; \vec v^{(2)} \\ 
|            &amp; | 
\end{bmatrix}$</p>
<p>则
$f(\vec b) = 
\begin{bmatrix}
V(V^T V)^{-1} V^T
\end{bmatrix}
\vec b 
$</p>
<p>将 $\vec b$ 投影到由 $v^{(1)}$ 和 $v^{(1)}$ 定义的子空间中</p>
<p>(每太听懂)</p>
<p>$X \theta = [X (X^T X)^{-1} X^T] y = X [(X^T X)^{-1} X^T y] \implies \underbrace{ \theta = (X^T X)^{-1} X^T y}_{\text{Normal Equation}}$</p>
<a class="post-dummy-target" id="分解"></a><h2>分解</h2>
<p>原向量v(圆) =&gt; 映射后向量Av(椭圆)
A: 旋转#1，缩放，旋转#2</p>
<a class="post-dummy-target" id="特征分解"></a><h3>特征分解</h3>
<p>旋转#1，复数缩放，旋转#1$^{-1}$</p>
<a class="post-dummy-target" id="svd"></a><h3>SVD</h3>
<p>旋转#1，实数缩放，旋转#2$</p>
<p>旋转操作：正交矩阵</p>
<p>缩放操作：对角矩阵</p>
<a class="post-dummy-target" id="谱"></a><h2>谱</h2>
<p>谱：特征值的集合</p>
<a class="post-dummy-target" id="性质"></a><h3>性质</h3>
<p>$\sum$特征值$ = \sum$迹</p>
<p>$\prod$特征值$ = $行列式值</p>
<a class="post-dummy-target" id="参考"></a><h2>参考</h2>
<p><a href="https://katex.org/docs/supported.html" target="_blank">$\KaTeX$ 文档</a></p></div><div class="post-footer" id="post-footer">
    <div class="post-info">
        <div class="post-info-line">
            <div class="post-info-mod">
                <span>本文于 2020-07-23 更新</span>
            </div>
            <div class="post-info-license"></div>
        </div>
        <div class="post-info-line">
            <div class="post-info-md"></div>
            <div class="post-info-share"><span><a href="//www.linkedin.com/shareArticle?url=https%3a%2f%2fdiraclee.gitee.io%2f2020%2f07%2fcs229_lecture2%2f&amp;title=cs229%20%e7%ac%ac%e4%ba%8c%e8%8a%82%20%28%20TA%20%29" target="_blank" title="分享到 LinkedIn">
            <i class="fab fa-linkedin fa-fw"></i>
        </a><a href="//service.weibo.com/share/share.php?url=https%3a%2f%2fdiraclee.gitee.io%2f2020%2f07%2fcs229_lecture2%2f&amp;appkey=&amp;title=cs229%20%e7%ac%ac%e4%ba%8c%e8%8a%82%20%28%20TA%20%29" target="_blank" title="分享到 Weibo">
            <i class="fab fa-weibo fa-fw"></i>
        </a></span></div>
        </div>
    </div>

    <div class="post-info-more">
        <section><span class="tag">
                        <a href="https://diraclee.gitee.io/tags/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/"><i class="fas fa-tag fa-fw"></i>&nbsp;机器学习</a>&nbsp;
                    </span><span class="tag">
                        <a href="https://diraclee.gitee.io/tags/cs229/"><i class="fas fa-tag fa-fw"></i>&nbsp;cs229</a>&nbsp;
                    </span></section>
        <section>
            <span><a href="javascript:window.history.back();">返回</a></span>&nbsp;|&nbsp;<span><a href="https://diraclee.gitee.io">主页</a></span>
        </section>
    </div>

    <div class="post-nav"><a href="https://diraclee.gitee.io/2020/07/design_pattern/" class="prev" rel="prev" title="设计模式"><i class="fas fa-angle-left fa-fw"></i>设计模式</a>
            <a href="https://diraclee.gitee.io/2020/07/cs229_lecture3/" class="next" rel="next" title="cs229 第三节">cs229 第三节<i class="fas fa-angle-right fa-fw"></i></a></div>
</div><div class="post-comment"></div>
    </article></div>
            </main><footer class="footer">
    <div class="copyright"><div class="copyright-line">由 <a href="https://gohugo.io/" target="_blank" rel="external nofollow noopener noreffer">Hugo</a> 强力驱动 | 主题 - <a href="https://github.com/dillonzq/LoveIt" target="_blank" rel="external nofollow noopener noreffer">LoveIt<i class="far fa-heart fa-fw"></i></a>
        </div>

        <div class="copyright-line"><i class="far fa-copyright fa-fw"></i><span itemprop="copyrightYear">2020</span><span class="author" itemprop="copyrightHolder">&nbsp;<a href="https://diraclee.gitee.io" target="_blank">Dirac Lee</a></span>&nbsp;|&nbsp;<span class="license"><a rel="license external nofollow noopener noreffer" href="https://creativecommons.org/licenses/by-nc/4.0/" target="_blank">CC BY-NC 4.0</a></span></div>
    </div>
</footer></div><a href="#" class="dynamic-to-top" id="dynamic-to-top" data-scroll>
            <span>&nbsp;</span>
        </a><script src="/js/lib/jquery/jquery.slim.min.js"></script><script src="/js/lib/lazysizes/lazysizes.min.js"></script><script src="/js/lib/smooth-scroll/smooth-scroll.polyfills.min.js"></script><script>window.scroll = new SmoothScroll('[data-scroll]', {speed: 300, speedAsDuration: true});</script><link rel="stylesheet" href="/css/lib/katex/katex.min.css"><script src="/js/lib/katex/katex.min.js"></script><script defer src="/js/lib/katex/auto-render.min.js"></script><link rel="stylesheet" href="/css/lib/katex/copy-tex.min.css"><script defer src="/js/lib/katex/copy-tex.min.js"></script><script defer src="/js/lib/katex/mhchem.min.js"></script><script>
        document.addEventListener("DOMContentLoaded", function () {
            renderMathInElement(document.body, {
                delimiters: [
                    { left: "$$", right: "$$", display: true },
                    { left: "\\(", right: "\\)", display: false },
                    { left: "\\[", right: "\\]", display: true },{ left: "$", right: "$", display: false },]
            });
        });
    </script><script src="/js/blog.min.js"></script></body>
</html>